Background: Benchmarking medical decision support algorithms often struggles due to limited access to datasets, narrow prediction tasks, and restricted input modalities. These limitations affect their clinical relevance and performance in high-stakes areas like emergency care, complicating replication, validation, and improvement of benchmarks. Methods: We introduce a dataset based on MIMIC-IV, benchmarking protocol, and initial results for evaluating multimodal decision support in the emergency department (ED). We use diverse data modalities from the first 1.5 hours of patient arrival, including demographics, biometrics, vital signs, lab values, and electrocardiogram waveforms. We analyze 1443 clinical labels across two contexts: predicting diagnoses with ICD-10 codes and forecasting patient deterioration. Results: Our multimodal diagnostic model achieves an AUROC score over 0.8 in a statistically significant manner for 357 out of 1428 conditions, including cardiac issues like myocardial infarction and non-cardiac conditions such as renal disease and diabetes. The deterioration model scores above 0.8 in a statistically significant manner for 13 out of 15 targets, including critical events like cardiac arrest and mechanical ventilation, ICU admission as well as short- and long-term mortality. Incorporating raw waveform data significantly improves model performance, which represents one of the first robust demonstrations of this effect. Conclusions: This study highlights the uniqueness of our dataset, which encompasses a wide range of clinical tasks and utilizes a comprehensive set of features collected early during the emergency after arriving at the ED. The strong performance, as evidenced by high AUROC scores across diagnostic and deterioration targets, underscores the potential of our approach to revolutionize decision-making in acute and emergency medicine.

An electrocardiogram (ECG or EKG) is a medical test that measures the heart's electrical activity. ECGs are often used to diagnose and monitor a wide range of heart conditions, including arrhythmias, heart attacks, and heart failure. On the one hand, the conventional ECG requires clinical measurement, which restricts its deployment to medical facilities. On the other hand, single-lead ECG has become popular on wearable devices using administered procedures. An alternative to ECG is Photoplethysmography (PPG), which uses non-invasive, low-cost optical methods to measure cardiac physiology, making it a suitable option for capturing vital heart signs in daily life. As a result, it has become increasingly popular in health monitoring and is used in various clinical and commercial wearable devices. While ECG and PPG correlate strongly, the latter does not offer significant clinical diagnostic value. Here, we propose a subject-independent attention-based deep state-space model to translate PPG signals to corresponding ECG waveforms. The model is highly data-efficient by incorporating prior knowledge in terms of probabilistic graphical models. Notably, the model enables the detection of atrial fibrillation (AFib), the most common heart rhythm disorder in adults, by complementing ECG's accuracy with continuous PPG monitoring. We evaluated the model on 55 subjects from the MIMIC III database. Quantitative and qualitative experimental results demonstrate the effectiveness and efficiency of our approach.

Deep learning provides an excellent avenue for optimizing diagnosis and patient monitoring for clinical-based applications, which can critically enhance the response time to the onset of various conditions. For cardiovascular disease, one such condition where the rising number of patients increasingly outweighs the availability of medical resources in different parts of the world, a core challenge is the automated classification of various cardiac abnormalities. Existing deep learning approaches have largely been limited to detecting the existence of an irregularity, as in binary classification, which has been achieved using networks such as CNNs and RNN/LSTMs. The next step is to accurately perform multi-class classification and determine the specific condition(s) from the inherently noisy multivariate waveform, which is a difficult task that could benefit from (1) a more powerful sequential network, and (2) the integration of clinical notes, which provide valuable semantic and clinical context from human doctors. Recently, Transformers have emerged as the state-of-the-art architecture for forecasting and prediction using time-series data, with their multi-headed attention mechanism, and ability to process whole sequences and learn both long and short-range dependencies. The proposed novel multi-modal Transformer architecture would be able to accurately perform this task while demonstrating the cross-domain effectiveness of Transformers, establishing a method for incorporating multiple data modalities within a Transformer for classification tasks, and laying the groundwork for automating real-time patient condition monitoring in clinical and ER settings. Deep learning has revolutionized medical signal processing and is able to outperform traditional Electrocardiogram (ECG) analysis for cardiac diagnostics (Smith et al., 2019). ECG waveforms, which depict the electrical activity of the heart, are a time series representation of the heart's voltage. Different ECG machines can have multiple leads, each representing different directions of cardiac activation (Park et al., 2022).

The electrocardiogram (ECG) is routinely used in hospitals to analyze cardiovascular status and health of an individual. Abnormal heart rhythms can be a precursor to more serious conditions including sudden cardiac death. Classifying abnormal rhythms is a laborious process prone to error. Therefore, tools that perform automated classification with high accuracy are highly desirable. The work presented classifies five different types of ECG arrhythmia based on AAMI EC57 standard and using the MIT-BIH data set. These include non-ectopic (normal), supraventricular, ventricular, fusion, and unknown beat. By appropriately transforming pre-processed ECG waveforms into a rich feature space along with appropriate post-processing and utilizing deep convolutional neural networks post fine-tuning and hyperparameter selection, it is shown that highly accurate classification for the five waveform types can be obtained. Performance on the test set indicated higher overall accuracy (98.62%), as well as better performance in classifying each of the five waveforms than hitherto reported in literature.

The emergence of deep learning networks raises a need for algorithms to explain their decisions so that users and domain experts can be confident using algorithmic recommendations for high-risk decisions. In this paper we leverage the information-rich latent space induced by such models to learn data representations or prototypes within such networks to elucidate their internal decision-making process. We introduce a novel application of case-based reasoning using prototypes to understand the decisions leading to the classification of time-series data, specifically investigating electrocardiogram (ECG) waveforms for classification of bradycardia, a slowing of heart rate, in infants. We improve upon existing models by explicitly optimizing for increased prototype diversity which in turn improves model accuracy by learning regions of the latent space that highlight features for distinguishing classes. We evaluate the hyperparameter space of our model to show robustness in diversity prototype generation and additionally, explore the resultant latent space of a deep classification network on ECG waveforms via an interactive tool to visualize the learned prototypical waveforms therein. We show that the prototypes are capable of learning real-world features - in our case-study ECG morphology related to bradycardia - as well as features within sub-classes. Our novel work leverages learned prototypical framework on two dimensional time-series data to produce explainable insights during classification tasks.

In intensive care units (ICU), electrocardiogram (ECG) waveforms show diverse variationsunder different patients' physical conditions.In general, physicians can diagnose patients efficientlyby detecting any disorder of heart rate or rhythm and any change in the morphological pattern of ECG data,which contain underlying semantics.To help physicians better analyze ECG data in a fairly short time,it is essential to develop a novel method for mining semantics from ECG patterns.This paper is the very first time to characterize ECG patterns by using Prefix Scalable Pattern Tree (PSP-Tree).Comparing with similar currently existing methods, PSP-Tree can mine significant semantics,such as scalability, temporality and hierarchy over ECG patterns.We conduct extensive experiments on real ECG data set which are obtained from PhysioBank Community and Beijing No.3 People Hospital.The experiment results show that our method performs more feasibly and effectively than other related work.

We examine the performance of Total Variation (TV) smoothing for processing of noisy Electrocardiogram (ECG) recorded by an ambulatory device. The TV smoothing is compared with traditionally-used band-pass filtering using ECG with artificially added noise, as well as with real-world noise obtained during physiological monitoring. The fundamental difference between TV smoothing and traditional band-pass filtering is that TV smoothing allow preserving sharp slopes in the ECG, while traditional smoothing dampens them. Since the QRS complex represents a feature with steep slopes, the TV smoothing is a better choice ECG filtering. We found that TV smoothing outperforms traditional filtering on ECG signals recorded under different conditions and can be used as effective filtering tool in popular QRS detection algorithms.

We examine the use of hidden Markov and hidden semi-Markov models for automatically segmenting an electrocardiogram waveform into its constituent waveform features. An undecimated wavelet transform is used to generate an overcomplete representation of the signal that is more appropriate for subsequent modelling. We show that the state durations implicit in a standard hidden Markov model are ill-suited to those of real ECG features, and we investigate the use of hidden semi-Markov models for improved state duration modelling.

We examine the use of hidden Markov and hidden semi-Markov models for automatically segmenting an electrocardiogram waveform into its constituent waveform features. An undecimated wavelet transform is used to generate an overcomplete representation of the signal that is more appropriate for subsequent modelling. We show that the state durations implicit in a standard hidden Markov model are ill-suited to those of real ECG features, and we investigate the use of hidden semi-Markov models for improved state duration modelling.

We examine the use of hidden Markov and hidden semi-Markov models forautomatically segmenting an electrocardiogram waveform into its constituent waveform features. An undecimated wavelet transform is used to generate an overcomplete representation of the signal that is more appropriate for subsequent modelling. We show that the state durations implicitin a standard hidden Markov model are ill-suited to those of real ECG features, and we investigate the use of hidden semi-Markov models for improved state duration modelling.